= Scope of this document =

This document covers all testing activities for the
Linux kprobe feature for MIPS arch.

After reading this document, a user should be able to:
 * Set up the environment to run all tests.
 * Run all tests and log results.
 * Understand what all tests are testing.

== Definitions and acronyms ==
 * k-xxx - sequential numbers for kprobes modules.
 * u-xxx - sequential numbers for kprobes user programs
   test cases.

= Prepare/Run Methodology =

== Target preparation ==
  * None.

= Test Suite =
== k-001: Simple kprobe with probe point to the kernel function do_fork ==

This test verifies that kprobe handler can debug specified address
successfully and ensures that pre handler and post handlers are
getting executed.

Steps:
 1. Build the kernel as per the Readme.txt document.
 2. Insert the module using command "insmod k-001.ko"
 3. Run any shell command (e.g ls, cat etc)

Pass Criteria:
 * The following lines should be shown.

1.insmod k-001.ko
Inserting the kprobe for do_fork

Address where the kprobe is
going to be inserted - 80045908

2.ls (Any shell command)

Before hook in do_fork
This is the Kprobe pre
handler for instruction at 80045908
Stack Dump :
Call Trace:
 [<c00114c4>] k_001_before_hook+0x5c/0x94 [k_001]
 [<80045908>] do_fork+0x0/0x274
 [<802031b4>] kprobe_exceptions_notify+0x34c/0x3ec
 [<80203678>] notifier_call_chain+0x38/0x70
 [<8014f670>] opost+0xe8/0x230
 [<80202b7c>] do_break+0x94/0xc8
 [<80203554>] do_page_fault+0x2e8/0x3d4
 [<80203398>] do_page_fault+0x12c/0x3d4
 [<80023030>] handle_bp_int+0x24/0x34
 [<801528ac>] write_chan+0x0/0x400
 [<8014966c>] tty_ldisc_deref+0xcc/0x110
 [<80027a20>] sys_fork+0x0/0x2c
 [<80027eac>] _sys_fork+0x34/0x40
 [<80045908>] do_fork+0x0/0x274
 [<8002c5f0>] stack_done+0x20/0x40
 [<8002c5f0>] stack_done+0x20/0x40

The Registers are:
Cpu 0
$ 0   : 00000000 10008401 00000002 2ac5e5f0
$ 4   : 00000012 7fe44970 878aff30 00000000
$ 8   : 00000000 80000010 80027a20 ffffffec
$12   : ffffffff 00000000 00000001 0000005d
$16   : 00000000 7fe44a28 1000dc38 00000000
$20   : 1002a508 00000000 00000000 00000000
$24   : 00000000 00000000
$28   : 878ae000 878aff10 1002a4a8 80027eac
Hi    : 0000000c
Lo    : 00000005
epc   : 80045908 do_fork+0x0/0x274 ra    : 80027eac _sys_fork+0x34/0x40
Status: 10008403    KERNEL EXL IE
Cause : 10000024
PrId  : 00002d30

After hook in do_fork
This is the Kprobe post
handler for instruction at 80045908
Stack Dump :
Call Trace:
 [<c0011558>] k_001_after_hook+0x5c/0x94 [k_001]
 [<80040000>] recalc_task_prio+0x74/0x218
 [<802031a4>] kprobe_exceptions_notify+0x33c/0x3ec
 [<80203678>] notifier_call_chain+0x38/0x70
 [<c00114e0>] k_001_before_hook+0x78/0x94 [k_001]
 [<80202b7c>] do_break+0x94/0xc8
 [<80023030>] handle_bp_int+0x24/0x34
 [<8014f670>] opost+0xe8/0x230
 [<80027a20>] sys_fork+0x0/0x2c
 [<80027eac>] _sys_fork+0x34/0x40
 [<80027eac>] _sys_fork+0x34/0x40
 [<8002c5f0>] stack_done+0x20/0x40
 [<8002c5f0>] stack_done+0x20/0x40

The Registers are:
Cpu 0
$ 0   : 00000000 10008401 00000002 2ac5e5f0
$ 4   : 00000012 7fe44970 878aff30 00000000
$ 8   : 00000000 80000010 80027a20 ffffffec
$12   : ffffffff 00000000 00000001 0000005d
$16   : 00000000 7fe44a28 1000dc38 00000000
$20   : 1002a508 00000000 00000000 00000000
$24   : 00000000 00000000
$28   : 878ae000 878afeb8 1002a4a8 80027eac
Hi    : 0000000c
Lo    : 00000005
epc   : c0000004 0xc0000004 ra    : 80027eac _sys_fork+0x34/0x40
Status: 10008402    KERNEL EXL
Cause : 10000024
PrId  : 00002d30

3.rmmod k-001.ko
 Module exiting from do_fork

== k-002: Multiple kprobes with the same probe point for the kernel function
	  do_fork ==

This test verifies that multiple kprobes can be placed at the same address.
And pre handler and post handlers for each kprobe gets executed.

Steps:
 1. Build the kernel as per the Readme.txt document.
 2. Insert the module using command "insmod k-002.ko"
 3. Run any shell command (e.g ls, cat etc)

Pass Criteria:
 * The following lines should be shown.

1. #insmod k-002.ko

Inserting two kprobes at the same probe point

Address where the two kprobes are
going to be inserted - 80045908 and 80045908

2. #ls
  Before hook 2nd do_fork

  Before hook 1st do_fork

  After hook 2nd do_fork

  After hook 1st do_fork

3. #rmmod k-002.ko

 Before hook 2nd do_fork

 Before hook 1st do_fork

 After hook 2nd do_fork

 After hook 1st do_fork

 Module exiting 2 kprobes from do-fork


== k-003: Multiple kprobes with the different probe point for the kernel function
	    do_fork and sys_gettimeofday ==

This test verifies that multiple kprobes can be placed at different address location.
And also ensure that pre handler and post handlers are getting executed for each kprobe.

Steps:
 1. Build the kernel as per the Readme.txt document.
 2. Insert the module using command "insmod k-003.ko"
 3. Run any shell command which calls both sys_gettimeofday() and
    do_fork()(e.g time ls, time date etc)

Pass Criteria:
 * The following lines should be shown.

1. #insmod k-003.ko

Inserting two kprobes at differnt probe point

Address where the kprobes are
going to be inserted - 80045908  and  8004ccd8

2. #ls
 Before hook in do_fork

 After hook in do_fork

 #time ls
 Before hook in gettimeofday

 After hook in gettimeofday

 Before hook in do_fork

 After hook in do_fork

#rmmod  k-003.ko

 Before hook in sys_gettimeofday

 After hook in sys_gettimeofday

 Before hook in do_fork

 After hook in do_fork

 Module exiting 2 kprobes for do-fork and sys_gettimeofday


== k-004: kprobes with the fault handler for the kernel function sys_gettimeofday ==

This test verifies that kprobes can handle the fault if the fault occurs in the
kprobe handler module. And also ensures that user defined kprobe fault handler
is getting executed successfully.

Steps:
 1. Build the kernel as per the Readme.txt document.
 2. Insert the module using command "insmod k-004.ko"
 3. Run any shell command which calls sys_gettimeofday()
    (e.g time ls, time date etc)

Pass Criteria:
 * The following lines should be shown.

1. #insmod k-004.ko
Inserting the kprobes  for sys_gettimeofday

Address where the kprobe is
going to be inserted - 8004ccd8

2. #time date
Before hook in sys_gettimeofday
This is the Kprobe pre
handler for instruction at8004ccd8
Stack Dump:
Call Trace:
 [<8008309c>] do_wp_page+0x2cc/0x528
 [<c0013528>] k_004_before_hook+0x60/0xb4 [k_004]
 [<8004ccd8>] sys_gettimeofday+0x0/0xb0
 [<802031b4>] kprobe_exceptions_notify+0x34c/0x3ec
 [<80083f84>] __handle_mm_fault+0x620/0x8ac
 [<80203678>] notifier_call_chain+0x38/0x70
 [<80202b7c>] do_break+0x94/0xc8
 [<80203554>] do_page_fault+0x2e8/0x3d4
 [<80203398>] do_page_fault+0x12c/0x3d4
 [<800278c4>] do_signal+0x2d0/0x408
 [<80023030>] handle_bp_int+0x24/0x34
 [<80041578>] default_wake_function+0x0/0x20
 [<8004ccd8>] sys_gettimeofday+0x0/0xb0
 [<8002c5f0>] stack_done+0x20/0x40
 [<8004ccd8>] sys_gettimeofday+0x0/0xb0

The Registers are:
Cpu 0
$ 0   : 00000000 10008400 0000004e 2ac4da0c
$ 4   : 7fe44918 7fe44920 00000000 00000000
$ 8   : 00000000 80000010 8004ccd8 fffffff4
$12   : 00000018 00100100 00200200 00000000
$16   : 1002a708 ffffffff ffffffff 7fe44918
$20   : 7fe44920 00000080 7fe44910 7fe44928
$24   : 00000000 2ac4d9fc
$28   : 878ae000 878aff30 7fe449b8 8002c5f0
Hi    : 00000030
Lo    : 0007e7fb
epc   : 8004ccd8 sys_gettimeofday+0x0/0xb0 ra    : 8002c5f0
stack_done+0x20/0x40
Status: 30008403    KERNEL EXL IE
Cause : 10000024
PrId  : 00002d30

This is the Kprobe fault
handler for sys_gettimeodday
fault_handler:p->addr=0x8004ccd8
Stack Dump:
Call Trace:
 [<8004dc30>] __do_softirq+0x90/0x158
 [<c001366c>] k_004_fault_probe+0x5c/0xac [k_004]
 [<802031ec>] kprobe_exceptions_notify+0x384/0x3ec
 [<8004dc30>] __do_softirq+0x90/0x158
 [<80203678>] notifier_call_chain+0x38/0x70
 [<8004dd88>] do_softirq+0x90/0xbc
 [<80047b54>] printk+0x0/0x28
 [<802032d0>] do_page_fault+0x64/0x3d4
 [<80021f0c>] ll_ip7+0xc/0x14
 [<80047438>] release_console_sem+0xf0/0x370
 [<80047b54>] printk+0x0/0x28
 [<8002e1a8>] tlb_do_page_fault_1+0x108/0x110
 [<80047b54>] printk+0x0/0x28
 [<8006ddbc>] __print_symbol+0x0/0xc8
 [<c001355c>] k_004_before_hook+0x94/0xb4 [k_004]
 [<c001348c>] k_004_CPY_FROM_USER+0x24/0x60 [k_004]
 [<80047b54>] printk+0x0/0x28
 [<c001355c>] k_004_before_hook+0x94/0xb4 [k_004]
 [<8004ccd8>] sys_gettimeofday+0x0/0xb0
 [<802031b4>] kprobe_exceptions_notify+0x34c/0x3ec
 [<80083f84>] __handle_mm_fault+0x620/0x8ac
 [<80203678>] notifier_call_chain+0x38/0x70
 [<80202b7c>] do_break+0x94/0xc8
 [<80203554>] do_page_fault+0x2e8/0x3d4
 [<80203398>] do_page_fault+0x12c/0x3d4
 [<800278c4>] do_signal+0x2d0/0x408
 [<80023030>] handle_bp_int+0x24/0x34
 [<80041578>] default_wake_function+0x0/0x20
 [<8004ccd8>] sys_gettimeofday+0x0/0xb0
 [<8002c5f0>] stack_done+0x20/0x40
 [<8004ccd8>] sys_gettimeofday+0x0/0xb0

The Registers are:
Cpu 0
$ 0   : 00000000 30008400 00000000 00000061
$ 4   : 802a53f4 00000000 000001f4 00418937
$ 8   : 00004721 802afd90 00000002 10004000
$12   : ff000000 00ff0000 00000002 b6000000
$16   : 80047b54 c0014780 878afe80 802a53f4
$20   : 802a0000 00000000 00000000 802a0000
$24   : 802a796d 00ff0000
$28   : 878ae000 878afdb0 7fe449b8 c001355c
Hi    : 00000063
Lo    : 126e2500
epc   : c001348c k_004_CPY_FROM_USER+0x24/0x60 [k_004] ra    : c001355c
k_004_before_hook+0x94/0xb4 [k_004]
Status: 30008403    KERNEL EXL IE
Cause : 1000000c
PrId  : 00002d30
put_user : -EFAULT
CPY_FROM_USER

After hook in sys_gettimeofday
This is the Kprobe post
handler for instruction at 8004ccd8
Stack Dump:
Call Trace:
 [<c00135d8>] k_004_after_hook+0x5c/0x94 [k_004]
 [<802031a4>] kprobe_exceptions_notify+0x33c/0x3ec
 [<80203678>] notifier_call_chain+0x38/0x70
 [<80083f84>] __handle_mm_fault+0x620/0x8ac
 [<80202b7c>] do_break+0x94/0xc8
 [<80203678>] notifier_call_chain+0x38/0x70
 [<80023030>] handle_bp_int+0x24/0x34
 [<800278c4>] do_signal+0x2d0/0x408
 [<8004ccd8>] sys_gettimeofday+0x0/0xb0
 [<8002c5f0>] stack_done+0x20/0x40
 [<8002c5f0>] stack_done+0x20/0x40

The Registers are:
Cpu 0
$ 0   : 00000000 10008400 0000004e 2ac4da0c
$ 4   : 7fe44918 7fe44920 00000000 00000000
$ 8   : 00000000 80000010 8004ccd8 fffffff4
$12   : 00000018 00100100 00200200 00000000
$16   : 1002a708 ffffffff ffffffff 7fe44918
$20   : 7fe44920 00000080 7fe44910 7fe44928
$24   : 00000000 2ac4d9fc
$28   : 878ae000 878aff08 7fe449b8 8002c5f0
Hi    : 00000030
Lo    : 0007e7fb
epc   : c0000004 0xc0000004 ra    : 8002c5f0 stack_done+0x20/0x40
Status: 30008402    KERNEL EXL
Cause : 10000024
PrId  : 00002d30

# rmmod k-004.ko
Module exiting from sys_gettimeofday

== u-005: A user program Read a 1KB to the buffer from the file every time.
  	    and increments the count for each read ==
This will count number of read() called,and also this testcase will take care
of inserting and removing k-005 module before and after the read() completion.

== k-005: A kprobe with probe point to the kernel function sys_read to count
   	    total number of sys_read called for the user program u-005 ==

This kernel module will count,and also prints the number of times sys_read()
is  called for the user program read().

Steps:
 1. Build the kernel as per the Readme.txt document.
 2. Run the program as ./u-005
 3. Insertion of the module k-005.ko is done by the program u-005.c
 4. Program u-005  will come out by removing of the module k-005.ko

Pass Criteria:
* The following lines should be shown.

 1. #gcc u-005.c -o u-005

 2. #./u-005
read api test
Inserting the kprobe for sys_read	#program does insmod k-005.ko

Address where the kprobe is
going to be inserted - 80093750
kernel kprobe_read_cnt is 111
Module exiting from sys_read		#program does rmmod k-005.ko

user_read_count 100

== u-006: A user program which writes 1KB to file from the buffer every time.
 	    and increments the count for each write ==

This will count number of write() called,and also this testcase will take care of
inserting and removing k-006 module before and after the write() completion

== k-006: A kprobe with probe point to the kernel function sys_write to count
   	    total number of sys_write called for the user program u-006 ==

This kernel module will count,and also prints the number of times sys_write()
is  called for the user program write().

Steps:
 1. Build the kernel as per the Readme.txt document.
 2. Run the program as ./u-006
 3. Insertion of the module k-006.ko is done by the program u-006.c
 4. Program u-006  will come out by removing of the module k-006.ko

Pass Criteria:
* The following lines should be shown.

 1. #./u-006
write api test

Inserting the kprobe for sys_write	#program does insmod k-006.ko

Address where the kprobe is
going to be inserted - 800937f0
kernel kprobe_write_cnt is 100

Module exiting from sys_write		#program does rmmod k-006.ko

user_write_count 100

== u-007: A user program which calls malloc and free for 50 time==

This will count number of malloc() and free called ,and also this testcase
will take care of inserting and removing k-007 kernel module.

== k-007: Test for kprobe with probe points to the kernel functon __kmalloc
	    and kfree called for the user program u-007 ==

This test should verify that kprobe handler debug the particular address
successfully.And pre handler and post handler are getting executed and
This test should give the no. of malloc/free calls done to the kernel.

Steps:
 1. Build the kernel as per the Readme.txt document.
 2. Run the program as ./u-007
 3. Insertion of the module k-007.ko is done by the program u-007.c
 4. Program u-007 will come out by removing of the module k-007.ko

Pass Criteria:
* The following lines should be shown.

 1. # ./u-007
					#program does insmod k-007.ko

SCEN_1_1: u-malloc-free.c
u-malloc-free.c 109 Successful comparison for Thread Id = 1
u-malloc-free.c 132 Successful comparison for Thread Id = 1
u-malloc-free.c 109 Successful comparison for Thread Id = 1
.....
.....
u-malloc-free.c 132 Successful comparison for Thread Id = 12
u-malloc-free.c 109 Successful comparison for Thread Id = 12
u-malloc-free.c 132 Successful comparison for Thread Id = 12
%%%%%%%%%%%%%%%%%

kprobe_malloc_count:50 kprobe_free_count=50

%%%%%%%%%%%%%%%%%
u-malloc-free.c TEST PASS(0)

%%%%%%%%%%%%%%%%%%%%%%%%%%		#program does rmmod k-007.ko



kmalloc count is 2231
kfree count is 2267



%%%%%%%%%%%%%%%%%%%%%%%%%%
ktime: exiting...

== OVERHEAD MESURMENT OF KPROBES SUPPORT FOR THE MIPS ==

== u-008: A test program to measure time taken to execute gettimeofday()==
This test should give the time taken to execute gettimeofday
interms of seconds, nanoseconds and microseconds for each call.

== k-008: A simple kprobe with a probe point to the kernel function
	  do_gettimeofday==
This test should verify that kprobe handler debug the particular
ddress successfully.And pre handler is getting executed.

To measure the overhead :
------------------------
1. First run the user program u-008 whithout inserting the kernel module k-008.
2. And for the second time insert the kernel module k-008,then execute the user
   program u-008
3. The difference will give the minimum overhead caused by the kprobes support

Steps:
 1. Build the kernel as per the Readme.txt document.
 2. This test needs to be executed two time
	2.1 To get the time taken to run the program without kprobes
		2.1.1 run ./u-008
	2.2 To get the time taken top run the program kprobes
		2.2.1 insert the kernel module k-008.ko as insmod k-008.ko
		2.2.2 Execute the user program as ./u-008
		2.2.3 Remove the kernel module k-007.ko as rmmod k-008.ko

Pass Criteria:
* The following lines should be shown.

Execute user program without kprobes
------------------------------------
 1. # ./u-008
gettimeofday was called 8645566 times per 10 sec: 1158.241699 nsec per call
gettimeofday was called 8645566 times per 10 sec: 1.158166 usec per call

Execute user program with the kprobes
-------------------------------------
 1. #  insmod k-008.ko

Inserting the kprobe at do_gettimeofday

Address where the kprobes are
going to be inserted - 8002867c

 2. # ./u-008
gettimeofday was called 3272488 times per 10 sec: 3059.596680 nsec per call
gettimeofday was called 3272488 times per 10 sec: 3.059140 usec per call

 3. #rmmod k-008.ko
Module exiting from gettimeofday


== u-009: A test program to measure time taken to execute gettimeofday()==
This test should give the time taken to execute gettimeofday
interms of seconds, nanoseconds and microseconds for each call.

== k-009: A simple kprobe with a probe point to the kernel function
	  do_gettimeofday==
This test should verify that kprobe handler debug the particular
address successfully.And pre handler and post handler are getting
executed

To measure the overhead :
------------------------
1. First run the user program u-009 whithout inserting the kernel module k-009.
2. And for the second time insert the kernel module k-009,then execute the user
   program u-009
3. The difference will give the overhead due to pre-handler and post-handler

Steps:
 1. Build the kernel as per the Readme.txt document.
 2. This test needs to be executed two time
	2.1 To get the time taken top run the program without kprobes
		2.1.1 ./u-009
	2.2 To get the time taken top run the program kprobes
		2.2.1 insert the kernel module k-009.ko as insmod k-008.ko
		2.2.2 Execute the user program as ./u-009
		2.2.3 Remove the kernel module k-009.ko as rmmod k-009.ko

Pass Criteria:
* The following lines should be shown.

Execute userprogram program without kprobes
-------------------------------------------
 1. # ./u-009
gettimeofday was called 8645561 times per 10 sec: 1158.357422 nsec per call
gettimeofday was called 8645561 times per 10 sec: 1.158282 usec per call

Execute userprogram program with the kprobes
-------------------------------------------
 1. #  insmod k-009.ko

Inserting the kprobe at do_gettimeofday

Address where the kprobes are
going to be inserted - 8002867c

 2. # ./u-009
gettimeofday was called 3243695 times per 10 sec: 3086.956543 nsec per call
gettimeofday was called 3243695 times per 10 sec: 3.086295 usec per call

 3. #rmmod k-009.ko
Module exiting from gettimeofday
